Faster Anomaly Detection via Matrix Sketching
نویسندگان
چکیده
We present efficient streaming algorithms to compute two commonly used anomaly measures: the rank-k leverage scores (aka Mahalanobis distance) and the rank-k projection distance, in the rowstreaming model. We show that commonly used matrix sketching techniques such as the Frequent Directions sketch and random projections can be used to approximate these measures. Our main technical contribution is to prove matrix perturbation inequalities for operators arising in the computation of these measures.
منابع مشابه
Streaming Anomaly Detection Using Randomized Matrix Sketching
Data is continuously being generated from sources such as machines, network traffic, application logs, etc. Timely and accurate detection of anomalies in massive data streams have important applications in preventing machine failures, intrusion detection, and dynamic load balancing. In this paper, we introduce a new anomaly detection algorithm, which can detect anomalies in a streaming fashion ...
متن کاملCorrections to “LD-Sketch: A Distributed Sketching Design for Accurate and Scalable Anomaly Detection in Network Data Streams”
In this article, we describe the corrections to our paper “LD-Sketch: A Distributed Sketching Design for Accurate and Scalable Anomaly Detection in Network Data Streams” published at IEEE INFOCOM 2014. We also clarify the complexity issue raised by some readers. 1 Corrections to Lemmas and Theorems
متن کاملSeparation Between Anomalous Targets and Background Based on the Decomposition of Reduced Dimension Hyperspectral Image
The application of anomaly detection has been given a special place among the different processings of hyperspectral images. Nowadays, many of the methods only use background information to detect between anomaly pixels and background. Due to noise and the presence of anomaly pixels in the background, the assumption of the specific statistical distribution of the background, as well as the co...
متن کاملHyperspectral Anomaly Detection via Background Estimation and Adaptive Weighted Sparse Representation
Anomaly detection is an important task in hyperspectral imagery (HSI) processing. It provides a new way to find targets that have significant spectral differences from the majority of the dataset. Recently, the representation-based methods have been proposed for detecting anomaly targets in HSIs. It is essential for this type of method to construct a valid background dictionary to distinguish a...
متن کاملThe detection of 11th of March 2011 Tohoku's TEC seismo-ionospheric anomalies using the Singular Value Thresholding (SVT) method
The Total Electron Content (TEC) measured by the Global Positioning System (GPS) is useful for registering the pre-earthquake ionospheric anomalies appearing before a large earthquake. In this paper the TEC value was predicted using the singular value thresholding (SVT) method. Also, the anomaly is detected utilizing this predicted value and the definition of the threshold value, leading to the...
متن کامل